Papers by S M Masrur Ahmed
MoMentS: A Comprehensive Multimodal Benchmark for Theory of Mind (2025.findings-emnlp)
Copied to clipboard
Emilio Villa-Cueva, S M Masrur Ahmed, Rendi Chevi, Jan Christian Blaise Cruz, Kareem Elzeky, Fermin Cristobal, Alham Fikri Aji, Skyler Wang, Rada Mihalcea, Thamar Solorio
| Challenge: | MoMentS is a benchmark designed to assess the ToM capabilities of multimodal large language models (LLMs) in short films. |
| Approach: | They introduce a benchmark to assess the ToM capabilities of multimodal large language models (LLMs) through realistic, narrative-rich scenarios presented in short films. |
| Outcome: | The proposed benchmark features long video context windows and realistic social interactions that provide deeper insight into characters’ mental states. |